Testing of a device is normally an integral part of the design, manufacture and maintenance of the device. Testing is routinely employed during the design of a new device to establish reliability and an operational capability of the device. In manufacture, testing establishes and/or quantifies operability of the device as well as facilitates yield improvements through failure diagnosis. Once the device is deployed, testing helps to maintain the device by detecting potential failures and diagnosing the cause or causes of failures of the device or its components.
In most cases, testing comprises performing a set or sequence of tests on the device under test (DUT). Such a set of tests or test suite typically employs one or more pieces of test equipment. Together, the test equipment and the test suite are referred to as a testing system. In some cases the testing system comprises a complex, integrated and highly automated system. In other cases, the testing system is only loosely integrated and many, if not all, of the tests are performed manually. In yet other cases, a testing subsystem is built into the DUT and functions as a built-in self-test for the DUT. As used herein, the term ‘testing system’ includes all such testing systems including a built-in testing subsystem of the DUT.
However, while testing systems vary widely according to the DUT and the application of the testing system, at some basic level all testing systems generally attempt to determine whether a given DUT is GOOD or BAD. A GOOD DUT is one that passes the tests administered by the testing system while a BAD DUT is one that fails one or more of the tests. Since DUTs generally are made up of one or more components, a secondary goal of many testing systems is to diagnose which component or components of the DUT is responsible the DUT being BAD. Thus, many testing systems include some form of a diagnostic system that performs a component or subsystem level diagnosis of failures encountered during testing. A DUT with a built-in test subsystem also may feature a built-in self-diagnostic capability.
For example, a given DUT typically includes a variety of components. Such components include, but are not limited to, integrated circuits, electrical components, battery subsystems, mechanical components, electrical buses, wiring components, and wiring harnesses. Any one or more of these components may fail and cause the failure of the DUT. The role of the diagnostic system portion of the testing system is to attempt to pinpoint the component or components that most likely caused the encountered failure or failure mode of the DUT.
Typically, diagnostic systems of testing systems either employ a fault tree approach to produce a diagnosis or attempt to extract a diagnosis from a combination of results of functional tests used to verify DUT functionality. In simple terms, a fault tree is a flow chart of tests to perform and related diagnoses. Tests are performed at each of a series of decision blocks of the flow chart. At each decision block, results of the test performed determine which of a plurality of branches leading out of the decision block is to be taken. Branches either lead to additional decision blocks or to stop blocks. Stop blocks give a diagnosis and represent an end point of the fault tree. Typically the stop blocks are annotated with the diagnosis (e.g., a name of a failed component) or with “no diagnosis is possible” indication. Thus, when a stop block is reached, the annotation gives the diagnosis. In general, only enough tests necessary to reach a stop block giving a diagnosis are performed when a fault tree is employed. Typically, fault trees are designed to mimic a decision-making process followed by a skilled test technician.
On the other hand, diagnostic systems based on the results of functional tests typically require that all tests of a predetermined set or test suite be performed before an attempt at a diagnosis can be made. Moreover, many such diagnostic systems employ a model or models of the DUT. As used herein, a model-based diagnostic system is defined as a diagnostic system that renders conclusions about the state or failure mode of the DUT using actual DUT responses to applied functional tests that are compared to expected responses to these tests produced by a model of the DUT. Often, the model is a computer-generated representation of the DUT that includes specific details of interactions between components of the DUT.
Selecting a suite of tests to perform is an important step in using model-based diagnostic testing systems. Performing too few tests or performing the wrong tests can lead to an inability to arrive at a diagnosis or even an incorrect diagnosis. In other words, a poorly designed test suite may not provide a particularly effective or reliable diagnosis of a DUT failure. In particular, for a given test suite, simply adding tests may not improve the diagnostic efficacy of the test system. Thus, often the problem is how to choose additional tests to add to the suite to increase diagnostic efficacy.
On the other hand, performing too many tests is potentially costly in terms of time and test equipment needed to arrive at a diagnosis. Moreover, some of the tests may be redundant and add little if anything to the diagnostic accuracy of the test system. In particular, many tests performed on a DUT can take an appreciable amount of time to perform and/or require the use of expensive equipment. If performing such tests does not increase the diagnostic efficacy of the testing system, the tests might just as well be eliminated. In such situations, the problem is one of determining which tests of a test suite may be eliminated without significantly reducing diagnostic efficacy.
Accordingly, it would be advantageous to be able to examine a test suite to determine what tests might be added to improve the diagnostic efficacy of the testing system. Likewise, it would be useful to be able to determine which tests in a test suite are redundant or have little added diagnostic value and, therefore, can be safely eliminated from the suite. Such abilities would address a longstanding need in the area of model-based diagnostic testing systems.